Papers with Large Audio-Language Models
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension (2024.acl-long)
Copied to clipboard
Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, Jingren Zhou
| Challenge: | Existing benchmarks for audio-centric interaction have impeded advancements in this field . AIR-Bench evaluates LALMs' ability to understand audio signals and interact with humans . |
| Approach: | They propose a benchmark to evaluate the ability of large audio-language models to understand audio signals . they use 19 tasks with approximately 19k single-choice questions to examine single-task ability . |
| Outcome: | The proposed framework evaluates the ability of large audio-language models to understand audio signals and interact with humans in the textual format. |
Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) have recently unlocked audio dialogue capabilities, enabling direct spoken exchanges with humans. |
| Approach: | They propose to evaluate LALMs' open-ended audio dialogue ability in 3 general scenarios, 12 skills, 9 multilingual languages, and 4 categories of ambiguity handling. |
| Outcome: | The proposed benchmark assesses the open-ended audio dialogue ability for LALMs in 3 general scenarios, 12 skills, 9 multilingual languages, and 4 categories of ambiguity handling. |
When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are augmented with the ability to perceive audio, but their reliability when faced with conflicting inputs remains largely unexplored. |
| Approach: | They examine how LALMs prioritize information when presented with inconsistent audio-text pairs. |
| Outcome: | The proposed models display a significant bias toward textual input when presented with inconsistent audio-text pairs. |
Extending Audio Context for Long-Form Understanding in Large Audio-Language Models (2026.eacl-long)
Copied to clipboard
Yuatyong Chaichana, Pittawat Taveekitworachai, Warit Sirichotedumrong, Potsawee Manakul, Kunat Pipatanakul
| Challenge: | Prior work has introduced context-extension methods (e.g. YaRN) on unimodal LLMs, yet their application to LALMs remains unexplored. |
| Approach: | They propose a training-free, modality-decoupled extension method that modifies only audio token positions, leaving text positions intact to preserve the base LLM’s text capabilities. |
| Outcome: | The proposed method outperforms the original models across wide range of settings and provides significant performance improvement on long audio of unseen lengths. |
SpeakerSleuth: Can Large Audio-Language Models Judge Speaker Consistency across Multi-turn Dialogues? (2026.acl-long)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are a popular approach for evaluating speech quality, yet their ability to assess speaker consistency across multi-turn dialogues remains unexplored. |
| Approach: | They construct 1,818 human-verified evaluation instances across four datasets spanning synthetic and real speech, with controlled acoustic difficulty. |
| Outcome: | The proposed model performs better in comparing and ranking acoustic variants, demonstrating inherent acustic discrimination capabilities. |
Pardon? Evaluating Conversational Repair in Large Audio-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of large audio-language models focus on answer accuracy and robustness to acoustic perturbations, but they assume that inputs remain semantically answerable. |
| Approach: | They propose a repair-aware evaluation setting that explicitly distinguishes between answerable and unanswerable audio inputs. |
| Outcome: | The proposed evaluation setting distinguishes between answerable and unanswerable audio inputs. |
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models (2026.acl-long)
Copied to clipboard
Zirui Song, Qian Jiang, Mingxuan Cui, Mingzhe Li, Lang Gao, Zeyu Zhang, Zixiang Xu, Yanbo Wang, Guangxian Ouyang, Zhenhao Chen, Xiuying Chen
| Challenge: | a recent study evaluated large audio-language models against jailbreak attacks . a new benchmark is being developed to evaluate LAM safety against jailbreaking attacks based on temporal and semantic nature of speech . |
| Approach: | They propose a benchmark to evaluate LAM jailbreak vulnerabilities in adversarial audio prompts . they use a dataset of 1,495 adversarials to evaluate their performance . |
| Outcome: | The proposed benchmark evaluates state-of-the-art LAMs against jailbreak attacks . it demonstrates that even small, semantically preserved perturbations can reduce safety . |
Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are increasingly being deployed in real-world applications, yet their robustness against malicious audio injection remains underexplored. |
| Approach: | They quantitatively assess their vulnerabilities and resilience using metrics: the Defense Success Rate, Context Robustness Score, and Judgment Robustic Index. |
| Outcome: | The proposed models demonstrate significant performance disparities across four attack scenarios. |
AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives (2026.findings-acl)
Copied to clipboard
Yanxi Chen, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Xin Li, Peijie Qiu, Hao Wang, Xuanzhao Dong, Yujian Xiong, Anderson Schneider, Yuriy Nevmyvaka, Yalin Wang
| Challenge: | Large Audio-Language Models suffer from hallucinations, e.g., generating text not grounded in the audio input. |
| Approach: | They propose a framework to address hallucination problems in large audio-language models . they use a preference dataset to test the model's accuracy . |
| Outcome: | The proposed model outperforms the latest SOTA methods in terms of performance and generalization. |
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on hallucination focus on text or vision, while few audio-oriented studies are limited in scale, modality coverage, and diagnostic depth. |
| Approach: | They propose a large-scale benchmark for evaluating hallucinations across speech, sound, and music. |
| Outcome: | The proposed model improves hallucination rate, yes/no bias, error-type analysis, and refusal rate. |
CliniCAST: Benchmarking Acoustic Grounding and Text Dominance in Medical Triage (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Large Audio-Language Models (LALMs) integrate acoustic capabilities into reasoning, yet whether they reliably ground clinical judgments in audible evidence remains unproven. |
| Approach: | They propose a benchmark that disentangles clinically meaningful acoustic cues from lexical content and speaker demographics. |
| Outcome: | Evaluating 5,856 synthetic samples across 12 disease conditions, the proposed model exhibits fragile acoustic grounding and pronounced "text dominance" failure mode. |